Audio-Oriented Multimodal Machine Comprehension via Dynamic Inter- and Intra-modality Attention

While Machine Comprehension (MC) has attracted extensive research interests in recent years, existing approaches mainly belong to the category of Reading task which mines textual inputs (paragraphs and questions) predict answers (choices or text spans). However, there are a lot MC tasks that accept audio input addition input, e.g. English listening comprehension test. In this paper, we target problem Audio-Oriented Multimodal Comprehension, its goal is answer questions based on given information. To solve problem, propose Dynamic Inter- Intra-modality Attention (DIIA) model effectively fuse two modalities (audio textual). DIIA can work as an independent component thus be easily integrated into models. Moreover, further develop Knowledge Distillation (MKD) module enable our multimodal accurately only either audio. As result, proposed approach handle various including: Listening single model, making fair comparisons possible between unimodal Experimental results analysis prove effectiveness approaches. First, boosts baseline models by up 21.08% terms accuracy; Second, under scenarios, MKD allows significantly outperform 18.87%, trained tested with data.